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tffifcftTS. fcJSU (6) SfCfc^T, I a I ti, X 
*5-aa©ffi*Hi*£U (d-d) (4, h;l/d 

[0 0 6 8] Rmin^^tifr^RckO fe^C^V^ 

w*©#%&tb-f set-?, 7YX«st5o ra— w 

I f(p, n-l)-f(p + D, n-1) | 

7) 

tfi&iLir&tZ, ffllp? (6) &tff£iL-?%frE5fr 

[0 0 6 9] W±OCt***AT, 0 8 ©^118 0 1 

(6) , (7) ttmrnicffirc-rmmpttiktiiL 
[0070] Kiai^w^K-rsfcibfc, hxh^i 
mm<Dpm%— -DtDmmictxmz&z.Wi'&icte, tiff 

■&tct4, Hi OfcjjVrj:?^ (6) ^fcttSrSSfc-rB 
[0 0 7 1] i=fc, J«jlTlK1*{**tttH-rs#ftfc L 

[0072] [»fr^^ h;i/*fflv^T, iSSi-elK Wf* 

¥JHl) ffl«*«»©:/ny*te$)"9JLTfre>, H*^5»J 
[0 0 7 3] ¥Wff 2) h;UO*«S3V--3e«tt 

[0 0 7 4] (6), (7) MMpt 

[0 0 7 5] 3.— *f-oaB-r*1H«J&^«i)3S^ 

[0 0 7 6]?I8 0 2T'ti> AHfSln-lfCg' 
[0 0 7 7] ^0B*»£O±«ffFfiffitD^T'(4, iSiST' 



fffcW 8-2 5 1 5 4 0 

fi1B«*ra-«rt*fc**U, 09{C^L/c<fc3fC In- 
1-h-e-ffiB p £0®^CDJt¥jaF f (p + D, n-1) (fcfc 
U DttRiinWT©S£*fcO''<* WW tOMtDiSJS 

0wU | D • D | ^RrainXRmin • • ( 

[0 0 7 8] ^SBfllTfcJ:, 4 «il¥2f 9 
I^CftoTl/^Ot', ^#^r4 71/- AC"t(C$i: 
«>, KH#fflfcnf4SCi:k:-rs. #IOi8 0 3T*«\ ^MB 

8 o zTflsjaftf^KiBMRffi^msnfc*^^^*^ 

^KHM8«*$tyKllB*ffi*UN (NiilW±(OSf») 
i:^IB-rsci:i:L, #18 0 4ftjtt? 0 r&Hi£n& 

</->cDT% #«4 0 6*«7fS. 
[0 0 7 9] #I8 0 4tt, iSiSi&f^EH^M^^CyE 

(Nniove- 1) ^HijCDE:P^fiiUN-(Niiiove-l)*4i:OP^ 

$ tl/c i $ , 7 V— I N-Nmove*4 £ I NOHfl^^S 

[0 0 8 0] #«S8 0 4©»7&* J^±fCiE<T£fc#- 
MB4 0 6^rHTt"i)o f LT> 0 4 <D#JiH 4 0 6 <DWT 

mm 4 o 7 (titty,, 

[0 0 8 1 ] ¥-JlR4 0 7«, Kn£j££lft0MR0fcti><a 
[0 0 8 2] ft*5, #Jl«4 0 7T*«, JKjSKlf^EM. fl( 

v\, s/c, s«fiaaft(Kiffl^(**i»Tw^-r*'!r-x 

[0 0 8 3] ¥(R4 0 7T-^L/cS^iSjaicL/-c^o 
OTtottlEl/feS/a *y hfiTK«Nshot*S!h:ftSo * 

ct% #)H4 0 8T% n±m&*®iE-?z>o 



(9) 



8-2 5 1 5 4 0 



[0 0 8 4] #M 4 0 8 Tit, *-f, #)H 4 0 4 T*&*b 
fc^a >y hSHkfcttbT, f?±(c£>g^:B#P^NSHk (k 

NSHk < Nshot 

) 

^ffl^ti^. (8) RftfRiL-? zm-Brlcte. 
Pifltfi/sy hftT&BflNshotfc&S.fcdl;:, m%.Mm%: 
[0 0 8 5] &*5, *niMmT*lZ, i/a >y hSTKffi^ 

[0 08 6] mm 4 0 8 <Dm7'ik, ¥M 4 0 9 iC'MtSo 

yxi^&oj;?^ B*5aja*asrSo ux 

T#S£5fC, m3Lm&*BJE-fZo ^Jl«4 0 9 

[0 0 8 7] ^1H4 0 9Kfe»5#l#Wfc»f£*EI 1 3 

V (k) = LSHk/Tlen • • 

) 

KJ;oT?l£3iJgV (k) ItiRtbZo 
[0 0 9 1 ] 1 3 0 3 T°1,Z. :x— tf— #5&3*tii~> 

h SHk©3£iiK&^<D«*p-??&SSI«±ffifiIVmax 
(k) So d OjgjaiPSffiVmax (k) Oi&j£73 

*a£LtE«vnax (k) (c^x-rs^ffi^^enso 

T*»^LfcHS«*Viiax (k) tfr%:tyj&t> 
#*.6>ft£>o ££lc, ¥Hi4 0 7"Pi*^Lfcfi«rraJH<tt 

[0 09 2] [fiBtPtSSfKEm, ft&MftE 

¥IHl) fiH*HSttlEIH*fr** fro, igjgRjftEP^* 

V (k) SVmax (k) • • • 

) 

^fflv^tl^s. (i 0) 5£#j£irr£*§^c(i> a— 

•»f-tcrt^^jea^-a-S/c:46^, Sai^->a >y h-SHk 

s Q c©0HEaaa»4, #jri 3 0 5T'HtTfSo 

[00 9 6] (1 0) ^7b^itL^l/>it^lCt±, 0 1 5 
#/c¥IH1 3 0 9%*l7-rSo 



shotTfeWcfc &ftxS 
(8 

[00 8 8] #^ISfi0ljT'«:, 0 1 4 {CTjrTtfiJ© <fc 5 UV 
^3 -y h©$te«7U— AiE«*»j£l$HTstl©lffl&^ 

LTfr5, 5/3 y h±f*«r¥2St)WrHITlen-e^SS03S 
^fS, £<Dt%. (Tstl + Tlen) MPS© 1 ; XA#f§ 
ftt§„ Tstl£0.2#£:U Tlen«:0.8#(Cia 

itfsi:, l^fiSo'jXA^fggfe-rso 

[0 0 8 9] 01 3 <D¥MI 13 0 1 T'ti, ^Sttfiv'a 
•y hSHk©7Ix— AtfcLSHk (k(i 1 J£t±©SE8Jt) 

[0 0 9 0] ¥Hi 1 3 0 2W±, ®i»t>'3 7 hSHk 

1 enters <£ 9 (C N 

(9 

§$iS:^3 7 hftt, ^?PEHf£lt<0->3 >y Mclt^ 
T, iifiiPSfSVniax (k) ***^«fc:R£-rSo #J*- 

tf, 8{gjitc^-rSo 

[0 0 9 3] ¥)H2) iS3SW)^Era^^> fro, ftNf 
RBSflaEffSfc^Sfcv^a >y hT-ti, fi^EP«gfctt<7» 
3-yMcif<T, jSJS±ISfiiVniax (k) 

^-rs 0 mz-is, 2^jS(cts^-rs 0 

[0 0 9 4] #«B3) «3S«iffKBBtfil^lfflJBiHEIffl* 
i«7!f£tr5/a y Ft, S^EP^/cttO^s >y htii, ii 
JK±figffiVmax (k) *R| CffltCtS^-T So 

[0 0 9 5] ^JlSl 3 0 4t'ti, (9) ^T'^L/cB 
£jijav (k) *V ¥Ji«l 3 0 3-Pife£Lfc3*aLhEfi* 
Vmax (k) J^±tC*S^if d 

(10 

[00 9 7] -TT'tCxE^fcfc 3 (c, 1 3 0 5S, > 

fay h<Dm^$fm%&&L, n£.m&*miE-? 

[0 0 9 8] ->3 -y hJ3y?{C'JXA£&fc-&£/c46{c, 
is a y hOg^P^* rftfif»3 -y h<DSS(^J tc-r 
So ft/cU ^'>3 -y h%M^T*^^tCf|^-rSfci6 



(10) 



¥fffl¥ 8-2 5 1 5 4 0 



<fS« ^CTSMBl 3 0 5T?tt, ( 1 0 ) 5££rfiS§JEf 
% *y a y h CUfijM 0 aP^oa^NPW*, ffi© favhO 
2fgT-ifc3 2 xTlentClS^-r^o 
[0 0 9 9] ^IC, ff^iijao^73j£fCOV>TxE^ 

do) sKfcWfc-rs'a v h ©nw»#©s£jas 

t>\ ^JUl 3 0 3-P^L/tigjS±|5g{lVmax (k) J-XT 

20«fc^fC, m*1$ft(On£j&&VP (k) £jgjg±PSffiV 
max (k) WTOffiKS£Lft±"P, ¥5M *) 



r (k) *ftj&r«o fc/cu «¥»^©n*iaiavR 

(k) teVmax (k) %j8*.Tfc<fcV\ > UXA*« 
±g|5#£Aft3o 

[0 10 0] ^iM 0 gP^«9¥OS^JlJgVP (k) 
[0101] 
062] 



f VmaxCK) - ^4^- > Vmax (K) ©£ # 



VP CK) J 



(11) 



VR (k) tt, 

v fl cK)^ s \: vnK) 

I i en 

KJjoTifcn?**. (11), (12) 

[0 10 3] rt?g*ffiST?*ftv^>3 y h*fa>< 

ofrf8£LTt>J:i^W&fc:«\ ¥JH1 3 0 2O»T»fc 
¥!4 0 9*«7bTtJ:V\ Sfc* i/g y hOEft* 
^fflT^a^Tt <tV^*&Ktt, (9) ^ffl^-ftcS 
£i*«V (k) 4SKLTfeJ:V>, 

[0 1 0 4] 04 ©#IH 4 0 9 <0|S7fe, ^HS 4 1 0 Ic 
mtSo #JH4 1 0&&^M4 1 2S:T'#«, B$^J(0^3 
y hcoratDfflW^ffl^T, SWWS'a y h^rlK-&LT'>3 
yhfft^U — o©^gy h»fr&«»©l$3&*>J<Z> 

7 i/-Aii («*tf i##) 3^6*s«5»»ia«*a! 

<Dz/a y MfflOffiMfcffll^T, mMisa y h*&t&LT 
fay h»*f&R-T* 0 #fc, *->a y HS*»6— o-T 

<fcpk:irax-rao 0>J*tf, mmnWtt^tsz/a >y 

a y h*»&T5*&KH\ C©NPH5*^>3 y hp© 
[0 10 5] fitt8M**B£-r*fc*fctt, jtalLfcSB 

[0 10 6] 3MS4 1 0^P>¥HS4 1 2 <fc o T^fiSc L 

[0 10 7] ¥MS4 1 Oli, ->3«y haf^fiSt-r-s^s 

T'fe^o ^«S4 1 OfCfcntSJIttffj&lM^ia 1 6£ffl 

[0 10 8] ->3 <y Mirtl?©-^*©'*?, 1 O0» 3 
y hF«9<07U-^H^(i 1 O£D«31<D^m^:tO 0 00* 
If, A^il^-f 3v-3 y hT'fi, f^T©? U-Ail 



[0 10 2] 
[»3] 



x Tien 



(12) 



tiA«J^S»f*o Lfetf^T, ~>3>yhrt©— SB© 

TtJt, i/a y h rt©Nrepfe©:7 lx— Apj^ (WT, ft 

[0 1 0 9] ¥)W 16 0 1 Tli, v-3 y hSHk^ft^f 
SH«*8H£LT, ft£l$2llinMkIk.j, (ktilJ^ 
±©^$5, j 14 1 W±NrepEtT©SE«0 t^ft-To ft 

[ 0 1 1 0 ] ¥WR 1 6 0 1 ©»T», ftSBt3graH#% 
ffl^TB^^JomfKi^©^ y ->3 y h 

iOi/a y h^MKftiD'>3 7 h 

Sfi^JT'(±, iSB±©^cDfei:»)#{caa lT->3 y h 

[oi i i] mmp><Dftmc£tii£. moifimv^av 

%t§o liffi{C3Hn?)2 0C»^li**^jiOltlff^LTV^ 

(IXT. KrtWSfcnMO VkK)<Dffii$. (J^T, flit 
[0112] ttilfeJt^^ff : ft^^F^ H^«D»±« 



(11) 



RPM¥ 8-2 5 1 5 4 0 



»f»l*ffllllIH«T L * l# MB 0 shot £<±©M£* 

[oii3] ttiifejt^yfeff^ssje-rssffiirts 
m i : m vmffi-vw&Ltt ->3 7h < *&o«t a 

ft, Sf«WR(D->3 >y ho 0 17 fc^Lfc:/-;l/*M H 
cDJSR<D 2 o<D~> a >y h SHk-1, SHkOftStnttSIHIBtt I 
k-1, U I k, ITHi, 

I k-1, Ht>0»±i»J«fe ACDSiJ-&= 6 0 % 
I k-1, ltf <0»ih«lJffift B ©ffl£= 2 0 % 
I k-1, l*<Ofj$±Ml&&CcomS= 1 5% 

i k-i, i^com±m^D<oms= 5 % 

I k, Iff ©»lh«J5EeA©8iJ-&= 8 0 % 
I k, l(tK0»±«fiK€sCO8"J^'= 1 5 % 
I k. If ©»±«J«fe E ©fU£-= 5 % 

J6lt5»±i)SftA, C© I k-1, if ©W£-©fD= 7 
5 % 

«ji-T§»l±1«fi5cfeA, CCD I k. If ©fiJ£-©*TJ= 9 5 
% 

t&S©T\ S#lh*JS&JbtifiB±f*<Z>7 5%W 

[0 114] W2 : Rli;*a?f*©S/3-y h 

©il*© 2 0©~> 3 >y h SHk-K SHk©f^B$^RapjflB I 
k-1, K I k, lt?ti, 

I k-i. if co&itMf&&vcomS= 4 o % 

I k-1, if CDit±1ifi5cfeWCDfiJ-&= 4 0 % 
I k-1. If ©ffMt#l/£feX©iiJ£-= 1 0% 

i k-i, if <D&±mf&&Y corns = 1 0% 

HSVk= <c I HS(c,SHk)>0H} 

3) 

HHVk= {c I HM(c,SHk)> 0H} 

4) 

JXTT'li, i/B v bmc09tmcOi/a «y h^rSHtop (toptt 

i w±©s») tmm-?% 0 #m 1 6 o 4 frz^m 1 6 

0 6 $T*«, C<Oi^a «y FSHtop^CDB^^JCDv-a >y 
h Zm^T, E C * T*© -> a -y h *^iifeJt^^#*?fl 

[0 119] 1 6 0 4 >y h-SHtop—SHtop 

H S C top.m = { c I cEHSVtopfl 



i k, if cD&±mf$.&z corns = 4 0 % 

I k, If <D#±1«^feWCDSiJ-a-= 3 5 % 

i k, if coB±mtft&xcoms= 1 0% 

1 k, If CO ffi±ffil&&Y com£> = 1 5% 

wa-rsiMijjgeiw, x, Y©ik-i,if©fy^©*rj= 

6 0% 

«31-r*!M»iJ8feW, X, Y©Ik, If ©fU^©fn=6 

0% 

tr*Z,cOT\ ^B-r«WRlS6AV1S^O6 0%U± 
[ 0 1 1 5 ] @ 1 6 ©¥J« 1 6 0 2 THi, f^S^P^Hi 

0 1 9{£^;-r<fc5fc:&£o 2fe(D7b— Alt© 

[0 1 16] HSU^tnRiS : \XmmSMmm^<Oy V- 
ABi I k, j tcoiy b— ANfK^<D 7 U 
-iMB&fccDiaT*, r[liL;fitHtc$>5iB*co)lif)SPM(D*g 

[0 1 17]f«1 603tt, »lt*USfe, MriJtJft&$ 

*JW!lt5. v' 3 -y h SHk(D^^raBt<D» 

itMi^&Cfc^T, R G B#8PgM© 5 1 2fecOtX )- 
^AHS (c, SHk) ^5t<J6?)o |S1«{C, Kl^iScDtX 
h^AHM (c, SHk) BIJRa*^* 

[0 118] 

(i 



(i 



±£aifeHSCtop, nu »J«iIfeHMCtop. ra (top, mfi 1 « 

±©»») fcnf-ECfcfcU *S*fflV^Ttt»-r*o 
[0 12 0] 
[&4] 



H S V top+i n - n HS V topfm J 



• • • » 



C15) 



HMC top»m = {c I cGHMV topHHM V IojH-i H ~ O HMVlo^m } 



(is; 



1 6 0 5 rstitttSfeHSCtop, mfcidWiifeHM 



(12) 



ftmW- 8-2 5 1 5 4 0 



[0121] 
C»5] 



AMC ( k » top.m) = 



£ H8(c.SHk)-f- X HMU,SHk) 

GeHSCtop.m ' GSlMCtop.m 

L HS(cSHk) h Z HM(cSHk) 

CEHSVk OSMVk 



(17) 



#iwi 6 o em nm&ttm£ift<DfitiL*m.'?&. 

AMC (k.top.m) SSshoU top^k^top+m 

) 

t>\ •?'<T<DklCftlsTf8.iL-?%t%, v's-yhSHtop 

~sHtop+«*^iifejtsp*fl : *»ije"rsi:** , r« 

{C, -r^TOktC^LT (18) S^ifTS^&fC 
(i, mfC 1 ^rSLTA^^JSl 6 0 4fcMSo 

<^*§-a-{c«> ^mm 1607 icmtSo 
co i 22] *mB&mvit. «^jo2o<Dj/3y kd 

5 0 £t\ ¥Mi 6 0 7T% (17) 3tvmm.Lrz 



(1 8 



«ilfetb^AMC (k.k.l) „ AMC (k+l,k,l) (ktitopJ-X 
±top+nJWT©S») *tHS-TSo M#AMC (k.k.l) H: 
T2OC0>'3 -y hSHk, SHk+l©f#±«5ffifei:lkH;jlfe 

*fc h5©^3 >y hSHk©f^BJ£BJB«i£K:i5i&SSy 

^a-To fS#AMC (k+l,k,l) (i, T2 0«D>'3>y h 
SHk, SHk+10Stitttiifei:ii!lttjife*\ t£<D~>3>ybSH 

k+iof^«ffflraiB«tt>fci5i6«fij^j ^a-r 0 

[0 1 2 3] 3MS 1 6 0 8Tti, ili&WjiifeJt^Og'h 



SS (k) =min (AMC(k.k.l), AMC(k+l ,k, 1)) 



(1 9 



) 

*tWU >^3>yhSHk, SHk+l<0^<Dfci6©g$feSfc 

[oi2 4]?ii 6 0 9tii, &m&stm$kW%ffirc 

"f ~>3 -y hSHtop~SHtop+m-lt. %r<D?K(DzS3 y hSHto 

SS (kmin) ^SS (k) . top^k^top+nrl 

) 

*ffifc?SS (kmin) ^5)<46, 02 1 (a) RXf (b) <D 
<y hgp(D*«<t^^-r o (02 1 (a) &tf (b) T*« 

i o^j-r^o 

[0 12 5] &*5, v-3 <y bg¥£*46 3 7? 



3 -y h(DP^O^©M$tJSSS (top) ~SS (top+m-1) <D 



(2 0 



fctf-pfr*,, Lfctf oT> #^U$8s#Ji:g&3 
Tjffi^rfflV^T, ->3 -y hg^^T&J:^,, 

[0 12 6] 3=fc, #JR1 6 0 6<Dm. ¥JH 1 6 0 7 
JiS-TK:, Wfflfelt^ftff^idfc-rs/a -y b<D*«SHk+ 

[0 12 7] 04<D¥lll4 1 0<D|?7^ #HH4 1 1 \C 
mts. ¥JP4 1 1 T1i> £->3 >y hpjb>e>— ofoOW 

(£, ->s >y bff<D$fe^v'3 -y hOftSgffltth ->3>yhB¥ 



[0 1 2 8] *«s, ¥JW4 1 1 Ttt, »#»H«*j8W 
Lfc*>\ g->a <y hSP*>P»— 3fo©MiMIlRL 

[0 129]fl4 1 1 <DWT'&. ¥WH4 1 2(Cjlt? 0 
3MH4 1 2t±, v'a y h^cD!(f?S%^-r^/ci60jaa 
T-feO, ¥JW4 1 1 Tg&LfcgBtflStffl&fcini&lI-f 

[0 1 3 0] *mMWiT"lis is B v hS¥{c-g-$ni.^a 
•y h*9SifeJ*4s*fl : *iB8fc , ro -T^:^^, ~>a -y bgp 
%mi£?%&i<'3 >y htc. fUl 60 7T'*i6/cf|±« 

[0131] t^±«3ife> ->a -y hS¥tc« 




(13) 



1#Rfl¥ 8-2 5 1 5 4 0 



[0 1 3 2] Sfc, H^OAPXiaa^jiti, ^3 7h© 

[0 13 3] ¥M4 1 2<D#5F7SL ¥JM4 0 2tcM-5 0 

l±> ¥WM4 l 2 ^UtTtmc, ¥JW4 l KD*l7tt{c^ 
MR 4 0 2lcRoTt<t>/\ 
CO 1 3 4] J^±2>\ 01CWSlOttIi5(DK 

ustWTfe o,@2 o^m 2 o i <d&hb 
[0135] tg&mftissmv&Ttit. 0 2 ©3Mh 202 

£jltrf So ¥IH2 0 2 Tfi, 0 1 tc&tt^g! 1 <DSt» 
a5^fci - f J ^-7-VX^S«l^VTR2^$i)fflILT> Stt 

£2ftfcHiW£0 1 ©BMftffiBStll 4fCfeKMPEG 

[0 13 6] fcJSU *HSg^J©¥IH 2 0 1 ti\ SiS^L 

[0 13 7] &33 s MLfcMP E Gj3ftT*&$fc-??>!& 
g«&< , J P E G^Sft£rofl6©fiEIS^=STJEE«LT 

©73 & ft 

[0 13 8] «TTIi« 02K*5ttS#jlI2 03©lft 

«s&frc*s» 0 1 ©sg 2 oswa 7 <Dtemn»n±9& 

[0 13 9] 02 3tis 0 1 iCl3tt%>m2<Dmm$k7(D 
[0 1 4 0] ¥H1 2 3 0 1 "CHi, 8Mfc^2Fffi«>a*?* 

[0 14 1] 4P5(W)»flF**J&4: LTF*3^ffi 

g L fcV tCli, 0 4tD¥llS4 0 5fr6¥MR4 0 81: 

[0 14 2] ±WllCltmA£tf3o2>o Lfctfo 



[0 1 4 3] • SH#fflSfclEIWT\ H£j&&*mtttmc 
m<-tZ>fzlst><D. ¥JH4 0 5 

<D, ¥MS 4 0 6 
&><D, ¥1W 4 0 8 

■$?>lC, B*^{C#$ft?>rt^£DB#P^W^^tft^»]t)fcV^ 
^-&{Ctt, 04cD¥JW4 0 9lCioTML/i HJXi* 

s^sikiiswrj *ss?-rft{f<fcv^ wmm^mc, mv 

[0 14 4] T#3f£ttS&3ft^©^£ / >L^Oji 
fct/ fCt±^ 0 4©¥JW4 1 0^e>¥/ll4 1 2i:T?<D 

[ 0 1 4 5 ] 0 2 3 ©¥JIS 2 3 0 2 t'tt, 3M1 2 3 0 1 
[0 1 4 6] *4B, 0 4{C^Lrc^S^)SaS©j}5tft 

¥IH2 3 0 1 ojffiw«fc:a&t>-e , Tay6ftff ^\ tc 
t7iif, ¥02 3 0 1 ©a&Ktc ry xam^SWW 
*^*ftftv«^tcii, 0 4 ©¥J« 4 0 9 fcHfrT 

^V^-a-tCli, 0 4CD¥JI14 0 5fr%¥ll4 0 8 ^T^- 

ntT-r^^g{i^<> ra$#aw?gtti»«j ^-&*ft^ 

V^-a-fCti. 0 4cr>¥JH4 1 0*->?.¥|ll4 1 2^T'^ 

[0 14 7] £fc, *^figCTT'«, S^L/c^#t5aS 
J*ftl»«*H 1 ©S&2cDft»a7±T-S£-rSW&^L 
fcTbV 774'M-^-6 t^^ftTV^flil©ttga± 
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(54) VIDEO SUMMARIZING METHOD 
(57)Abstract: 

PURPOSE: To summarize automatically video by solving a problem that a desired 
content cannot be grasped when the video summarized by the user is viewed so as to 
provide efficiently a content among the operation of an object with respect to the 
method summarizing and displaying the video to be used for a device supporting 
retrieval, editing, processing and glance of the video. 

CONSTITUTION: The frame image of a video signal received by a video disk device 1 
or a VTR 2 is received by a frame memory 3, a 1 st computer 5 gives a control signal to 
the frame memory 3 to receive a frame image and the video is summarized by 
processing a time series frame image and date are stored in a file server 6. A 2nd 
computer 7 calls a summarized video from the file server 6 on request by the user and 
reproduces the video. 
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CLAIMS 



[Claim(s)] 

[Claim 1] The image epitome approach which calculates the similarity between the 
frame images of the image used as the candidate for an epitome, detects the long 
duration similar section which is the time amount section when an image similar over 
beyond the time amount on which it decided beforehand continues, and reproduces 
said long duration similar section quickly compared with the time amount sections 
other than said long duration similar section. 

[Claim 2] The image epitome approach according to claim 1 which divides two frame 
images into two or more subregions, respectively, and calculates the similarity of said 
two frame images using the similarity of said subregion. 

[Claim 3] The image epitome approach which detects the high-speed operation 
section which is the time amount section which the body on a screen moves quickly 
compared with other time amount sections, and reproduces said high-speed operation 



section late from the frame image of the image used as the candidate for an epitome 
compared with the time amount sections other than said high-speed operation 
section. 

[Claim 4] The image epitome approach according to claim 3 of detecting the 
high-speed operation section in the field in which the frame image was decided 
beforehand. 

[Claim 5] The frame image IM of time series (M is the one or more natural numbers) is 
sampled from the image used as the candidate for an epitome. The pixel PN in the 
specific frame image IN (N is the natural number of 1 [or more ] and under M) The 
brightness difference between the pixels around [ corresponding to said pixel PN ] 
pixel PN+1 is calculated in the following frame image IN+1. The image epitome 
approach according to claim 3 or 4 of asking for said pixel PN to which said brightness 
difference exceeds the threshold thetaWI set up beforehand per each frame image, 
and asking for the high-speed operation section using said pixel PN. 
[Claim 6] The frame image IM of time series (M is the one or more natural numbers) is 
sampled from the image used as the candidate for an epitome. The pixel PN in the 
specific frame image IN (N is the natural number of 1 [or more ] and under M) The 1st 
brightness difference which is a brightness difference between the pixels around said 
pixel PN is calculated. Said pixel PN The 2nd brightness difference which is a 
brightness difference between the pixels around [ corresponding to said pixel PN ] 
pixel PN+1 is calculated in the following frame image IN+1. Per each frame image The 
image epitome approach according to claim 3 or 4 of asking for said pixel PN to which 
said 1st brightness difference becomes less than thetaWI threshold set up 
beforehand, and said 2nd brightness difference exceeds said threshold thetaWI, and 
asking for the high-speed operation section using said pixel PN. 

[Claim 7] The image epitome approach according to claim 3 or 4 of asking for the 
high-speed operation section using a motion vector. 

[Claim 8] Claim 1 which packs the frame image of the image used as the candidate for 
an epitome for every contents, divides into two or more shots, and prepares a 
minimum in the display time length of said shot according to the contents of said shot 
thru/or claim 7 are the image epitome approach of a publication either. 
[Claim 9] The image epitome approach which carries out sequential playback of said 
shot while pack the frame image of the image used as the candidate for an epitome for 
every contents, and it divides into two or more shots, determining reproduction speed 
so that the boundary of said shot and the rhythm of the period set up beforehand may 
have correlation, and reminding a rhythm. 

[Claim 10] The image epitome approach according to claim 9 of reminding a rhythm 
using a sound. 

[Claim 1 1] between the time amount Tstl which set up beforehand the frame image 
which interrupts playback at intervals of the time amount Tien set up beforehand, and 



0 



is displayed at the time — displaying (Tlen+Tstl) — the image epitome approach 
according to claim 9 of determining reproduction speed so that the rhythm made into 
a period may be generated and said rhythm may have the boundary of a shot, and 
correlation. 

[Claim 12] The image epitome approach which packs the frame image of the image 
used as the candidate for an epitome for every contents, divides into two or more 
shots, calculates the rate upper limit which is the criteria of reproduction-speed 
decision according to the contents of the image, reproduces some [ at least ] shots at 
the rate of below said rate upper limit, determines the reproduction speed of said shot 
so that it may become below the time-amount length upper limit that is a threshold 
which the display-time length of said shot set up beforehand, and carries out the 
sequential playback of the shot. 

[Claim 13] Pack the frame image of the image used as the candidate for an epitome 
for every contents, and it divides into two or more shots. According to the contents of 
said image, the rate upper limit which is the criteria of reproduction speed decision is 
calculated. When reproduction speed required in order to display said whole shot by 
the conventional-time length Trhy who set up beforehand becomes said below rate 
upper limit The reproduction speed of said shot is determined so that the display time 
length of said shot may turn into said conventional-time length Trhy. When 
reproduction speed required in order to display said whole shot by said 
conventional-time length Trhy exceeds said rate upper limit The image epitome 
approach which sets up the display time length of said shot the twice of said 
conventional-time length Trhy, determines the reproduction speed of said shot so 
that the reproduction speed between Trh(ies) in the first half of said display time 
length may become said below rate upper limit, and carries out sequential playback of 
the shot of time series. 

[Claim 14] The image epitome approach according to claim 12 or 13 of setting a rate 
upper limit as a large value compared with the shot which does not include said long 
duration similar section at the shot which calculates the similarity between frame 
images, detects the long duration similar section which is the time amount section 
when contents similar over beyond the time amount on which it decided beforehand 
continue, and includes said long duration similar section. 

[Claim 15] The image epitome approach according to claim 14 which divides two frame 
images into two or more subregions, respectively, and calculates the similarity of two 
frame images using the similarity of said subregion. 

[Claim 16] The image epitome approach according to claim 12 or 13 of setting a rate 
upper limit as a small value compared with the shot which does not include said 
high-speed operation section at the shot which detects the high-speed operation 
section which is the time amount section which the body on a screen moves quickly 
compared with other time amount sections, and includes said high-speed operation 



section from the frame image of the image used as the candidate for an epitome. 
[Claim 17] The image epitome approach according to claim 16 of detecting the 
high-speed operation section in the field in which the frame image was decided 
beforehand. 

[Claim 18] The frame image IM of time series (M is the one or more natural numbers) 
is sampled from the image used as the candidate for an epitome. The pixel PN in the 
specific frame image IN (N is the natural number of 1 [or more ] and under M) The 
brightness difference between the pixels around [ corresponding to said pixel PN ] 
pixel PN+1 is calculated in the following frame image IN+1. The image epitome 
approach according to claim 16 or 17 of asking for said pixel PN to which said 
brightness difference exceeds the threshold thetaWI set up beforehand per each 
frame image, and asking for the high-speed operation section using said pixel PN. 
[Claim 19] The frame image IM of time series (M is the one or more natural numbers) 
is sampled from the image used as the candidate for an epitome. The pixel PN in the 
specific frame image IN (N is the natural number of 1 [or more ] and under M) The 1st 
brightness difference which is a brightness difference between the pixels around said 
pixel PN is calculated. Said pixel PN The 2nd brightness difference which is a 
brightness difference between the pixels around [ corresponding to said pixel PN ] 
pixel PN+1 is calculated in the following frame image IN+1. Per each frame image The 
image epitome approach according to claim 16 or 17 of asking for said pixel PN to 
which said 1st brightness difference becomes less than thetaWI threshold set up 
beforehand, and said 2nd brightness difference exceeds said threshold thetaWI, and 
asking for the high-speed operation section using said pixel PN. 

[Claim 20] The image epitome approach according to claim 16 or 17 of asking for the 
high-speed operation section using a motion vector. 

[Claim 21] The image epitome approach which divides the frame image of the image 
used as the candidate for an epitome into two or more shots, unifies said shot similar 
using correlation between said shots of time series, regards it as a shot group, 
chooses from said shot group the partial dynamic image which is a frame image of two 
or more time series, and carries out sequential playback of said partial dynamic image. 
[Claim 22] The image epitome approach according to claim 21 which searches for the 
description which is common from all the shots contained in a shot group, processes a 
partial dynamic image based on said description, and carries out sequential playback 
of the partial dynamic image after processing. 

[Claim 23] The image epitome approach according to claim 22 which searches for the 
description which is common from all the shots contained in a shot group, processes a 
partial dynamic image based on said description, reduces the frame image of the head 
of the partial dynamic image after processing, and indicates by list on a screen. 
[Claim 24] Claim 21 which detects the shot of the time series by which the similar 
background and the similar photographic subject are photoed, and is made into a shot 



group thru/or claim 23 are the image epitome approach of a publication either. 
[Claim 25] Claim 21 which processes the image between representation space-time 
which is a frame image chosen from shots, divides into the motion area which is a field 
influenced of a change according the image between said representation space-time 
to a motion, and the quiescence field which are fields other than said motion area, 
creates the histogram of a color for every field, and searches for correlation between 
shots using the histogram of said color thru/or claim 24 are the image epitome 
approach of a publication either. 

[Claim 26] The image epitome approach according to claim 25 of processing a partial 
dynamic image in a shot group using the intersection of ******** which is a color 
which appears in a motion area, and the intersection of the quiescence configuration 
color which is a color which appears in a quiescence field. 

[Claim 27] Claim 21 which time amount length detects the shot which continues in 
time almost equally, and considers is a shot group thru/or claim 23 are the image 
epitome approach of a publication either. 

[Claim 28] Claim 8 which considers that the part continuously photoed in time with 
the video camera is a shot thru/or claim 27 are the image epitome approach of a 
publication either. 

[Claim 29] Claim 8 which considers that one scene of a scenario is a shot thru/or 
claim 27 are the image epitome approach of a publication either. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention summarizes the image which started the 
approach of supporting retrieval of an image, edit, processing, Hayami, etc., especially 
was stored in the video tape or the videodisk, and relates to the image epitome 
approach which gives playback or an indication. 
[0002] 

[Description of the Prior Art] In recent years, a computer etc. is applied and the 
research on the approach of supporting retrieval of an image, edit, processing, Hayami, 
etc. prospers. As the example, in order to grasp the contents of the image of video 
and a movie for a short time, there is the image epitome approach which reproduces 
the whole part and whole image of an image for a short time. 

[0003] As a conventional approach, the rushes reclaiming process is proposed the 
adjustable-speed reclaiming process (an Otsuji, Tomura, and "subjectivity evaluation 
of dynamic-image high-speed browsing" 1993 Institute of Electronics, Information and 



Communication Engineers spring convention, SD-9-3), and the whole shot the 
rapid-traverse reclaiming process, the contents compression display process 
approach of an image given in JP,4-237284,A, and the whole shot. In addition, a shot is 
the unit of the image often used in fields, such as image edit, and it is close to the 
smallest unit of the contents of an image. 

[0004] The conventional rapid-traverse reclaiming process is the image epitome 
approach which thins out a frame image in a fixed time interval, and is reproduced. 
[0005] The contents compression method of presentation of an image is the image 
epitome approach that judge the display significance of each frame image and a frame 
image with a higher significance spares much time amount for a display using the 
variation between the frame images of time series. For utilization, the part which 
judges the display significance of a frame image is important. 

[0006] The brightness variation of the pixel unit between contiguity frame images is 
sensitive to a motion of the body in a screen. Then, few places of a motion are flown 
and the method of considering that the brightness variation of this pixel unit is the 
above-mentioned display significance is proposed as the display significance judging 
approach used to see slowly the place with many motions. 

[0007] On the other hand, like [ the brightness variation of the frame unit between 
contiguity frame images is comparatively insensible to migration of the body within a 
shot, and / in case a shot changes ], when the luminance distribution inclination of the 
whole frame changes, a big value comes out. Then, the method of considering that the 
brightness variation of this frame unit is the above-mentioned display significance is 
proposed as the display significance judging approach used to see change of a shot 
carefully. 

[0008] An adjustable-speed reclaiming process is the image epitome approach which 
reproduces an image the conventional whole shot, controlling reproduction speed to 
make display time of each shot regularity. 

[0009] Moreover, a rushes reclaiming process is the image epitome approach which 
reproduces the head part of a shot one after another by standard speed the whole 
shot. 

[0010] Finally, it supplements with explanation about the shot. The part continuously 
photoed in time with one video camera is called a shot. A shot is close to the smallest 
unit of the contents of an image, as mentioned above. Moreover, the part connected 
by edit and the part which interrupted photography of a video camera serve as 
"change of a shot." 

[0011] However, it may be considered as an exception that the part from which the 
contents of an image change with camera actuation of a pan, a zoom, etc. is "change 
of a shot." In this case, compared with the case where an exception is not taken into 
consideration, a shot becomes close to the smallest unit of the contents of an image. 
[0012] As an approach of dividing an image into a shot automatically, the image 



change model method (Yamada, Fujioka, Kanamori, Matsushima, Sakauchi, the "scene 
change detection approach of an image including the edit effectiveness", multimedia, 
and image processing symposium'94) etc. is proposed. 
[0013] 

[Problem(s) to be Solved by the Invention] However, in the above-mentioned 
rapid-traverse reclaiming process, since it reproduced with constant speed regardless 
of the contents of an image, it saw on "subjectivity target and a part with quick 
reproduction speed and late partial" existed, contents grasp was difficult and the user 
had a technical problem called the fatigue and a cone. 

[0014] Moreover, in the above-mentioned contents compression method of 
presentation of an image, only the approach of using the brightness variation of the 
pixel unit between contiguity frame images, and the approach using the brightness 
variation of a frame unit were proposed as the judgment approach of display 
significance. Since the brightness variation of a pixel unit was unsuitable to subjective 
motion evaluation when the former is used, it saw on "subjectivity target and a part 
with quick reproduction speed and late partial" existed, grasp of the contents was 
difficult and the user had a technical problem called the fatigue and a cone. Moreover, 
since only the head frame image of each shot was displayed when the latter is used, it 
had the technical problem that it could not use to grasp the contents focusing on 
actuation of a photographic subject. 

[0015] The above-mentioned whole shot, with the adjustable-speed reclaiming 
process, since reproduction speed was decided by the time amount length of a shot, a 
user has not grasped the contents of some shots but it had the technical problem that 
it could not use to know the time flow of the contents included in an image. 
[001 6] It becomes important [ it is important to grasp the contents of each shot which 
is the unit of the contents, and ], in order to avoid an oversight of a shot that change 
of a shot can be predicted to know the time flow of the contents. However, the whole 
shot, with an adjustable-speed reclaiming process, if the case "where a shot changes 
in the part connected by edit and the part which interrupted photography of a video 
camera" is removed, change of a shot cannot be predicted. In the time of considering 
that the part photoed by the same camera actuation is the above-mentioned shot, 
and treating it, and the time of considering that one scene of a scenario is the 
above-mentioned shot, and treating it, change of a shot could not be predicted but it 
had the technical problem that an oversight of a shot occurred. 

[0017] With an image, the combination of two or more shots serves as a unit of 
contents higher order than a shot like a scene. The above-mentioned whole shot, with 
the adjustable-speed reclaiming process, since all the shots were reproduced little by 
little, the part of similar contents will be reproduced continuously and it had the 
technical problem that it could not use to see efficiently the part of different contents 
as much as possible. 



[0018] This invention solves the technical problem of the above-mentioned 
conventional technique, and it aims at offering the image epitome approach that a 
user can grasp the desired contents. 
[0019] 

[Means for Solving the Problem] In order to attain this purpose, the time amount 
section of the image containing the body which moves at high speed is detected, and 
it is regarded [ 1 st ] as the high-speed operation section. Moreover, the time amount 
section when an image similar over beyond fixed time amount continues is detected, 
and it is regarded as the long duration similar section. And the high-speed operation 
section is reproduced at a low speed compared with the time amount sections other 
than the high-speed operation section, and the long duration similar section is 
reproduced at high speed compared with the time amount sections other than the 
long duration similar section. Furthermore, a minimum is prepared for the display time 
length of each shot, after summarizing an image for every contents and dividing into 
two or more shots. 

[0020] The shot of time series is reproduced one after another, reminding the rhythm 
of the period set up beforehand, after summarizing an image for every contents and 
dividing [ 2nd ] into a shot. However, an upper limit is prepared for the display time 
length of each shot. Moreover, it is determined that reproduction speed will satisfy at 
least one side of the following two conditions. 

[0021] Rhythm conditions: The boundary of a shot and the rhythm of the period set up 
beforehand have correlation. 

[0022] Contents conditions : When the rate upper limit is set up so that it may 
guarantee "the part reproduced at the rate of below the rate upper limit which is the 
criteria of reproduction speed decision can surely grasp the contents", a part of [ at 
least ] reproduction speed of each shot becomes below a rate upper limit. 
[0023] The image for an epitome is divided into the 3rd at a shot. After it, using 
correlation between the shots of time series, a similar shot is unified and a shot group 
is created. And every one partial dynamic image is chosen from each shot group, and a 
partial dynamic image is reproduced one after another. 
[0024] 

[Function] Since the body which moves at high speed stops existing and each 
contents are displayed [ 1st ] beyond fixed time amount by these configurations, "it is 
a part with quick reproduction speed subjectively" disappears. Moreover, since an 
image changes one after another for a short time, "it is a part with slow reproduction 
speed subjectively" disappears. Therefore, fatigue of a user is mitigated compared 
with a conventional method. Moreover, since this approach reproduces the whole 
image, seeing subjectively and making it reproduction speed enter in tolerance, it can 
grasp contents centering on actuation of a photographic subject. 
[0025] To the 2nd, the part of the same contents is summarized to a shot and each is 



reproduced for a short time. Therefore, since change of a shot can be predicted when 
summarizing an image as satisfying rhythm conditions, it can see, without overlooking 
no shots. 

[0026] On the other hand, when summarizing an image as satisfying contents 
conditions, a user can grasp some of the contents at least in all shots. Since it is the 
shot which summarized the part of the same contents, the contents of all the shots 
can be grasped. 

[0027] In addition, since the part of the same contents is summarized, a shot is 
created and these are reproduced one after another, this means is used to know the 
time flow of the contents included in an image. However, since change of a shot 
cannot be predicted when not satisfying rhythm conditions, an oversight of a shot 
occurs. When not satisfying contents conditions, the contents of some shots cannot 
be grasped. It is desirable that it is satisfied with coincidence of rhythm conditions and 
contents conditions. 

[0028] Since the part of the contents to which time series is similar is summarized, a 
shot group is created and these parts are reproduced [ 3rd ] one after another, the 
part of different contents as much as possible can be seen efficiently. 
[0029] 

[Example] Hereafter, one example of this invention is explained, referring to a drawing. 
[0030] Drawing 1 is the whole image epitome equipment system chart in one example 
of this invention. In drawing 1 , 1 and 2 are the input devices of the image (it is 
hereafter called a processing-object image) used as a processing object, 1 is 
videodisk equipment and 2 is VTR. Moreover, 3 is a frame memory which captures the 
frame image of the video signal outputted from videodisk equipment 1 or VTR2. 4 is an 
image compression equipment which compresses the video signal outputted from 
videodisk equipment 1 or VTR2. 5 is the 1st calculating machine and controls 
videodisk equipment 1, VTR2, a frame memory 3, and the image compression 
equipment 4. Moreover, a control signal is sent to a frame memory 3, a frame image is 
captured, and an image is summarized by processing the frame image of time series. 6 
is a file server which memorizes the data sent from the image compressed with the 
image compression equipment 4, or the 1 st calculating machine 5, and a frame image. 
7 is the 2nd computer which calls an epitome image and a processing-object image 
from a file server 6, and reproduces the image according to a demand of a user. 
[0031] About the image epitome equipment constituted as mentioned above, 
actuation of the whole is explained using the flow chart shown in drawing 2 . 
[0032] In a procedure 201, an image is summarized because the 1st calculating 
machine 5 in drawing 1 processes the frame image of time series, controlling videodisk 
equipment 1 , VTR2, and a frame memory 3. 

[0033] In a procedure 202, controlling videodisk equipment 1, VTR2, and the image 
compression equipment 4, the 1st calculating machine 5 compresses the image and 



processing-object image which were summarized, and memorizes to a file server 6. 
[0034] In addition, not only the approach of compressing an image directly about the 
store method of an epitome image but other approaches are various idea ****. For 
example, as shown in drawing 3 , when an epitome image can be expressed by file 
format using the frame number of a processing-object image, the information on a 
frame number may be memorized to a file server 6 instead of an epitome image. 
Moreover, controlling videodisk equipment 1, VTR2, and a frame memory 3, the head 
frame image of each shot group mentioned later may be captured, and you may 
memorize to a file server 6 instead of an epitome image. However, it is equivalent to 
showing the epitome of an image to reduce the memorized head frame image in this 
case, and to indicate by list. 

[0035] In a procedure 203, according to a demand of a user, the 2nd calculating 
machine 7 calls an epitome image and a processing-object image from a file server 6, 
and reproduces the image. 

[0036] Below, the image epitome processing of the 1st computer 5 which is concrete 
actuation of the procedure 201 in drawing 2 is explained. 

[0037] Drawing 4 is the flow chart of one example of image epitome processing of the 
1st calculating machine 5 in drawing 1 . 

[0038] First, in a procedure 401, initialization of various parameters used for image 
epitome processing and a setup of a threshold are performed. Moreover, when 
calculating characteristic quantity, such as "the similarity between frame images", a 
high-speed operation pixel, a quiescence configuration color, ********, a quiescence 
common color, and a ** common color, in the procedure mentioned later, in order to 
use, the processing-object frame image In (n is the one or more natural numbers) is 
captured from a frame memory 3, and this frame image is memorized. 
[0039] Next, in a procedure 402, when it judges whether the image used as a 
processing object was completed and an image is completed, image epitome 
processing is ended. When the image is not completed, it progresses to a procedure 
403. 

[0040] In a procedure 403, it is considered that the following frame image In+1 of a 
current processing-object frame image is the new processing-object frame image In. 
And a control signal is sent to videodisk equipment 1 or VTR2, the processing-object 
frame image In after updating is reproduced, and the frame image In is captured from a 
frame memory 3. 

[0041] In a procedure 404, the tail of the shot which is the part continuously photoed 
in time with one video camera is detected. When the tail of a shot is detected, after 
considering that this shot is the new processing-object shot SHk (k is one or more 
integers), it progresses to a procedure 405. When the tail of a shot is not detected, it 
progresses to a procedure 405, without doing anything. In addition, after progressing 
to a procedure 405, it is that this procedure 404 is performed again, after 



incorporating the following processing-object frame image In+1. Therefore, when the 
tail of a shot is undetectable, after incorporating degree processing-object frame 
image In+1, detection of the tail of a shot will be tried again. 

[0042] In addition, you may consider that one scene of a scenario etc. is a shot. 
Therefore, all the "shots" used by the explanation after a procedure 404 may be reset 
on "one scene of a scenario." 

[0043] Moreover, the image change model method introduced by the Prior art is 
proposed as the detection approach of the tail of a shot. Furthermore, in this example, 
although the image is automatically divided into the shot, while a user looks at an 
image etc., you may divide. A procedure 404 may be skipped when the image is 
beforehand divided into the shot. 

[0044] After a procedure 405, even a procedure 408 is the image epitome approach 
using the following three assumptions built based on the subjectivity evaluation result 
of a rapid-traverse image, and reproduction speed is determined that it sees on 
"subjectivity target and reproduction speed will extinguish a quick part and late 
partial." 

[0045] - In the part (it is hereafter called the long duration similar section) which the 
image which carried out long duration resemblance follows, it is sensed that 
reproduction speed is slow. 

[0046] - In the part (it is hereafter called the high-speed operation section) of the 
image containing the body which moves at high speed, it is sensed that reproduction 
speed is quick. 

[0047] - When an image is summarized for every contents and it divides into a shot, it 
is sensed at a shot (it is called a short-time length shot) with short display time that 
reproduction speed is quick. 

[0048] In addition, the epitome image created by this approach will be called a section 
gear change epitome image. Moreover, in this example, the time amount sections 
other than the long duration similar section and the high-speed operation section will 
be called the standard section. Furthermore, the reproduction speed of the standard 
section is set as 4X. 

[0049] A procedure 405 is processing which detects the long duration similar section. 
This detection processing is performed because the image of 4X rapid-traverse 
playback samples the image of four-frame spacing and investigates the similarity 
between that image, since it consists of images of four-frame spacing. The concrete 
actuation in a procedure 405 is described using drawing 5 . 

[0050] In a procedure 501, the frame number of the processing-object frame image In 
is investigated first. Next, this frame image judges whether it is contained in the image 
sequences of four-frame spacing. Since the long duration similar section is detected 
only using the image of four-frame spacing, when judged with "It is contained in image 
sequences", in order to detect the long duration similar section, it progresses to a 
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procedure 502. When judged with It is not contained in image sequences , a 
procedure 405 is ended and it progresses to the procedure 406 of drawing 4 . 
[0051] In a procedure 502, the similarity S between frame image In-4 of the image 
sequences in front of one (n, n-4) is calculated by the processing-object frame image 
In and the above-mentioned image sequences. However, the count approach of the 
similarity between frame images was decided so that correlation with the motion 
which the user who looks at an epitome image can ignore subjectively might be lost. 
[0052] Here, the count approach of the similarity between frame images is described 
briefly. Although various kinds of approaches, such as chi2 assay (Nagasaka, Tanaka, 
"the method of detecting a scene change of a video work automatically", the 40th 
time national conference of Information Processing Society of Japan, 1Q-1990 [ 5 
or ]), can be considered In this example, a common color ratio method (a Yamada, 
Fujioka, Kanamori, Matsushima, and "examination of scene change detection approach 
of having observed common color for every subregion" television society technical 
report, September, 1993, Vol.17, No.55, pp 1-6), and the same approach are used. 
[0053] With the static image and the image only including the motion which can be 
disregarded subjectively, the distance which the body on a screen moves to 4 frame 
time on a screen should become very small compared with the width of face of a 
screen. According to experience of writers, this distance becomes 2% or less of the 
width of face of a screen. Then, as shown in drawing 6 , the frame image In is divided 
into Subregion R G. n) (however, j or more 1 16 or less integer). At this time, the 
histogram of a color hardly changes between a static image, the image In only 
including the motion which can be disregarded subjectively, and the subregions R (j, n) 
and R Q, n-4) where In-4 correspond. In addition, although the number of subregions 
was set to 16 in this example, it is not necessary to be necessarily 16. 
[0054] On the other hand, if change of a shot etc. occurs between the frame image In 
and In-4, between In and the subregion where In-4 correspond, the color which a 
photographic subject changes and constitutes subregion will change. Then, this 
similarity is calculated so that the similarity Sp (j, In, In-4) of subregion which 
corresponds according to the area of the color which appeared newly, and the color 
which disappeared between corresponding subregions may decrease. 
[0055] And the average of the frame image In and the similarity of the subregion 
where In-4 correspond is calculated, and it is regarded as the similarity S of a frame 
image (n, n-4). Namely, [0056] 
[Equation 1] 
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S(n,n-0= Z Sp ( j i In i In-4) ( 1 ) 



It calculates. 
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[0057] A procedure 503 is processing which updates the head frame number of the 
long duration similar section. The image in the long duration similar section is 
[ whether since it is mutually similar, the similarity S between the images of time 
series (n n-4) becomes more than threshold thetastill, and ] conditional expression. 
S(n-4, n) >=thetastill (2) 

**************. (2) When a formula is materialized, consider that (n-4) is the middle 
of the long duration similar section, and if the similar section edge Nb showing a head 
frame number is not set up, substitute (n-4) for Nb. 

[0058] A procedure 504 is processing which detects the candidate of the long 
duration similar section. It is [ whether the similarity of the image of four frame 
spacing becomes more than the above-mentioned threshold thetastill between the 
frame image of the similar section edge Nb, and the processing-object frame image In, 
and ] conditional expression. S(Nb+(M) x4, Nb+ix4) >=thetastill, and 1 <=i<= (n-Nb) / 
4 (3) 

**************. (3) When a formula is materialized, consider that between the frame 
image INb of a similar section edge and the above-mentioned processing-object frame 
images In is the candidate of the long duration similar section, and progress to a 
procedure 505. (3) Since the candidate of the similar section cannot be determined for 
a long time when a formula is not materialized, progress to the procedure 406 of 
drawing 4 . 

[0059] "The part which carries out loose image change" as shown in drawing 7 other 
than "the part which a similar image follows" is contained in the candidate who asked 
in the procedure 504. It is [ whether in a procedure 505, the similarity between a long 
duration similar section candidate's head frame image INb and other frame images 
becomes first more than threshold thetaratio, and ] conditional expression. S(Nb, 

Nb+ix4) >=thetaratio, 1 <=i<= (n-Nb)/4 (4) 

Next, it is [ whether when (4) types are materialized, this 
candidate's time amount length is long compared with the minimum time amount 
length Tstill of the long duration similar section, and ] conditional expression. n-Nb >= 

Tstill (5) 

However, the minimum time amount length Tstill in (5) types sets up 
at the time of activation of the procedure 401 of drawing 4 . 

[0060] When (4) and (5) types are materialized in coincidence, after considering that 
the candidate who asked in the procedure 504 is the long duration similar section, the 
procedure 405 described above is ended. 

[0061] (4) If the image after the processing-object frame image In is not processed 
when a formula is materialized and (5) types are not materialized, the candidate who 
asked in the procedure 504 cannot judge whether it is a part of long duration similar 
section. Then, a procedure 405 is ended, without doing anything. 

[0062] (4) When a formula is not materialized, consider that this candidate is "the part 



which carries out loose image change." Moreover, since (2) types are materialized, it 
is considered that a frame number (n-4) or subsequent ones is the candidate of the 
long duration similar section. (n-4) is substituted for the similar section edge Nb 
showing the head frame number of the candidate of the long duration similar section. 
[0063] In addition, in this example, the count approach of the similarity between frame 
images was decided so that correlation with the motion which the user who looks at 
an epitome image can ignore subjectively might be lost. However, since a user's 
subjectivity varies by the user, it may think as important a motion of the body which a 
user observes and may determine the count approach of similarity. 
[0064] Moreover, in this example, it asked for "the part which a similar image follows" 
using (3) and (4) types. That is, the similarity between the images of time series and 
the similarity on the basis of the head frame image of the similar section were used. 
However, since the same effectiveness is acquired even if it uses the approach on the 
basis of the frame image of the last of the similar section etc., "the part which a 
similar image follows" may be calculated using approaches other than (3) and (4) 
types. Moreover, "the part which a similar image follows" may be calculated only using 
(4) types. 

[0065] It progresses to a procedure 406 after procedure 405 termination of drawing 4 . 
A procedure 406 is processing which detects "the part of the image containing the 
body which moves at high speed." The concrete actuation in a procedure 406 is 
described using drawing 8 . 

[0066] A procedure 801 is processing which detects the pixel on the 
processing-object frame image In and the body which moves at high speed using 
image In-1 in front of that. The principle of this processing is explained using drawing 
9 . 

[0067] As shown in drawing 9 , when the body on a location p (however, p vector) 
carries out high-speed migration, within distance Lmin, it is assumed between frame 
image In-1 and In that the body of a basis does not already exist. At this time, by 
migration of a body, on In-1 The brightness f of the pixel of a location p (p, n-1) 
Brightness f of the pixel in the circle of the radius Lmin centering on the location p 
same on In (p+d) All the brightness differences between n) (however, vector in which d 
has the die length below Lmin) become thetawl or more thresholds set up beforehand. 
|f(p, n-1)-f(p+d, n) |>=thetaw1, |d-d|<=LminxLmin .. (6) 

It However, in (6) types, |a| expresses the absolute value of scalar quantity a, 

and (d-d) expresses the inner product of Vector d. 

[0068] Furthermore, a noise is removed by detecting only a larger body than the circle 
which makes Rmin a radius. As it considers that the same brightness field is the same 
body and was shown in drawing 9 , it is the brightness f of the pixel of a location p 
(p+D) on In-1. Conditional expression from which the brightness difference between 
n-1 (however, vector in which D has the die length below Rmin) becomes less than 



thetawl threshold |f(p, n-1)-f(p+D, n-1) |<thetaw1 and |D-D|<=RminxRmin (7) 
When ******(ing), it investigates whether (6) types are materialized in a location p. 
[0069] After detecting the pixel p which fills (6) and (7) types with the procedure 801 
of drawing 8 to coincidence based on the above thing, it considers that this pixel is "a 
pixel on the body which moves at high speed", and it is registered as a high-speed 
operation pixel. 

[0070] in addition, the case where pixel infanticide is carried out in order to simplify 
processing and 8pixelx — it may be considered that the average of the pixel for eight 
lines is one pixel Thus, in making one pixel represent the contents of two or more 
pixels, the area per pixel becomes large like a mosaic. In this case, as shown in drawing 
10 , you may consider that the pixel which fills only (6) types is a high-speed operation 
pixel. 

[0071] Moreover, the approach using the motion vector as an approach of detecting 
the body which moves at high speed can be considered. Like the following procedure, 
a high-speed operation pixel may be detected using a motion vector. 
[0072] After dividing into two or more blocks [example of procedure of detecting body 
which moves at high speed using motion vector] procedure 1 image, it asks for the 
motion vector for every block using the image of two sheets of time series. 
[0073] The block with which the magnitude of procedure 2 motion vector becomes 
more than constant value is searched for. 

It is considered that the pixel contained in the block searched for in procedure 3 
procedure 2 is a high-speed operation pixel. 

[0074] Furthermore, although the pixel in comparison with Pixel p was made into the 
pixel contained in a specific circle by (6) and (7) formulas, as shown in drawing 1 1 , it is 
good also as a pixel contained in the square containing Pixel p. 

[0075] Moreover, a high-speed migration pixel may be detected out of the pixel 
contained to the field assumed beforehand supposing the field which a user observes. 
For example, as shown in drawing 12 , it may assume that a user always observes near 
the center of a screen, and a high-speed migration pixel may be detected. 
[0076] In a procedure 802, the total of the high-speed operation pixel contained in 
frame image In-1 is investigated. And when the total of a high-speed operation pixel 
becomes a large value compared with threshold thetam set up in order to remove a 
noise, it is considered that between this frame image In-1 and following frame image In 
is a high-speed operation section candidate. 

[0077] If it moves by the result of subjectivity evaluation of a rapid-traverse image 
only for a moment even if the body which moves at high speed exists, it is ignored 
subjectively. Then, a procedure 804 excepts the section which became a candidate by 
momentary motion out of a high-speed operation section candidate. Although some 
implementation approaches of this processing are considered, they describe that one 
approach here. 



[0078] In this example, since the image of 4X rapid-traverse playback has been 
criteria, an image is summarized every four frames and it will be called a section unit. 
In a procedure 803, it judges whether the high-speed operation section candidate was 
detected by the procedure 802. When it judges with "It was detected", it supposes 
that it writes UN [ a section unit including this high-speed operation section 
candidate ] (N is one or more integers), and progresses to a procedure 804. Since a 
high-speed operation section candidate does not exist when it judges with "It is not 
detected", a procedure 406 is ended. 

[0079] A procedure 804 is processing which judges whether it continues more than 
the lower limit Nmove that the section unit including a high-speed operation section 
candidate set up beforehand. Moreover, it investigates whether all the section units 
between the section unit UN and section unit UN-(Nmove-l) *4 in front of the 
(Nmove-1) individual contain a high-speed operation section candidate. Next, when 
judged with "A high-speed operation section candidate is included", it is considered 
that between frame image IN-Nmove*4 and IN(s) is the high-speed operation section. 
[0080] The procedure 406 stated above is ended after termination of a procedure 804. 
And it progresses to a procedure 407 after termination of the procedure 406 of 
drawing 4 . 

[0081] A procedure 407 is processing which determines the image reproduction rate 
for a section gear change epitome image. First, the reproduction speed of the 
high-speed operation section is determined as 2X. Next, what does not include the 
high-speed operation section in the middle is taken out out of the long duration similar 
section, and the reproduction speed is determined as 8X. Furthermore, it considers 
that the remaining section which reproduction speed has not determined is the 
standard section, and the reproduction speed is determined as 4X. 
[0082] In addition, in a procedure 407, although the reproduction speed of the 
high-speed operation section, the standard section, and the long duration similar 
section was set as 2X, 4X, and 8X, respectively, as long as the reproduction speed of 
the high-speed operation section is slow compared with the standard section and the 
reproduction speed of the long duration similar section is quick compared with the 
standard section, you may set it as other values. Moreover, reproduction speed may 
be determined using display time like the case which reproduces the whole long 
duration similar section in 1 second. 

[0083] When an image is reproduced according to the reproduction speed determined 
in the procedure 407, display time becomes under the shot length lower limit Nshot 
set up beforehand at some shots. Then, reproduction speed is corrected in a 
procedure 408. 

[0084] In a procedure 408, the time amount NSHk required for playback (k is one or 
more integers) is first found from each shot SHk for which it asked in the procedure 
404. Next, it is [ whether it becomes under the shot length lower limit Nshot and ] 



conditional expression. NSHk < Nshot (8) 

(8) When a formula is materialized, consider that Shot SHk is a 
short-time length shot, and determine reproduction speed so that the display time of 
a shot may become the shot length lower limit Nshot. 

[0085] In addition, in this example, although the shot length lower limit was made into 
the fixed value, a shot length lower limit may be changed according to the contents, 
such as fineness of a texture. 

[0086] It progresses to a procedure 409 after termination of a procedure 408. A 
procedure 409 is a procedure for realizing the image epitome approach which 
reproduces the shot of time series one after another, reminding this "rhythm", after 
determining reproduction speed that the boundary of a shot and the rhythm of the 
period set up beforehand will have correlation. Reproduction speed is determined so 
that first an epitome image may have the rhythm of the period set up beforehand on 
the boundary of a shot. Next, holding a rhythm, reproduction speed is corrected so 
that a user can grasp the contents of each shot. In addition, the epitome image 
created in the procedure 409 will be called a rhythm presentation epitome image. 
[0087] The concrete actuation in a procedure 409 is described using drawing 13 . 
What is necessary is just to determine reproduction speed so that display time of for 
example, each shot may be made regularity in order to give the rhythm of the period 
beforehand set as the boundary of a shot. Moreover, as an approach of reminding a 
rhythm, various kinds of approaches, such as an approach of accustoming a sound on 
the boundary of a shot, and the approach of displaying a static image on the boundary 
of a shot, can be considered. 

[0088] At this example, like the example shown in drawing 14 , after displaying the 
head frame image of a shot between the quiescence time amount Tstl, the whole shot 
is indicated by rapid traverse by the rapid-traverse time amount Tien. The rhythm of 
spacing (Tstl+Tlen) occurs at this time. For example, if Tstl is made into 0.2 seconds 
and Tien is set as 0.8 seconds, the rhythm of 1 -second spacing will occur. 
[0089] In the procedure 1301 of drawing 13 t frame number LSHk (k is one or more 
integers) of the processing-object shot SHk is calculated. 

[0090] in a procedure 1302, the display time of the rapid-traverse part of the 
processing-object shot SHk turns into the rapid-traverse time amount Tien set up 

beforehand — as — V(k) =LSHk/Tlen (9) 

Reproduction speed V (k) is decided to be alike. 

[0091] In a procedure 1303, the rate upper limit Vmax (k) which is the criteria of the 
reproduction speed decision of the processing-object shot SHk is determined so that 
a user can grasp the contents of the processing-object shot SHk. The decision 
approaches of this rate upper limit Vmax (k) are various idea *♦**. For example, the 
reproduction speed determined in the procedure 407 is averaged in the 
processing-object shot SHk, and how to assign this value to the rate upper limit Vmax 



(k) can be considered. Moreover, the method of considering that the fixed value set up 
beforehand is Vmax (k) is also considered. Furthermore, the following method of 
having used the similar section and standard section and the high-speed operation 
section for a long time which was determined in the procedure 407 is also considered. 
[0092] At the shot which does not include the high-speed operation section, the rate 
upper limit Vmax (k) is set as a large value compared with the shot of only the 
standard section, including the [rate upper-limit decision approach using long duration 
similar section, standard section, and the high-speed operation section] procedure 1 
long-duration similar section. For example, it is set as 8X. 

[0093] At the shot which does not include the long duration similar section, the rate 
upper limit Vmax (k) is set as a small value compared with the shot of only the 
standard section, including the procedure 2 high-speed-operation section. For 
example, it is set as 2X. 

[0094] At a shot including both the procedure 3 high-speed-operation section and the 
long duration similar section, and the shot of only the standard section, the rate upper 
limit Vmax (k) is set as the same value. For example, it is set as 4X. 
[0095] It is [ whether in a procedure 1304, reproduction speed V (k) determined by (9) 
formulas becomes more than the rate upper limit Vmax (k) determined in the 

procedure 1303, and ] conditional expression. V(k) >=Vmax (k) (10) 

************** (10) When a formula is materialized, in order to make a user grasp 
the contents, extend the display time of the rapid-traverse part of the 
processing-object shot SHk, and correct reproduction speed. This correction 
processing is performed in a procedure 1305. 

[0096] (10) When a formula is not materialized, end the procedure 1309 stated above 
like the part of the shot SH1 of drawing 15 after determining the reproduction speed 
of the rapid-traverse part of Shot SHk as V (k). 

[0097] As already stated, a procedure 1305 is processing for making a user grasp the 
contents of the shot, extends the display time of a shot and corrects reproduction 
speed. 

[0098] in order to give a rhythm to a shot boundary — the display time of a shot — 
" — others — it is made integral multiple" of a shot. However, in order to reproduce 
each shot one after another for a short time, display time of the rapid-traverse part of 
each shot is shortened as much as possible. Then, the display time of the 
rapid-traverse part of the shot which is satisfied with a procedure 1305 of (10) types 
is set as 2xTlen which is twice other shots. 

[0099] Next, the decision approach of reproduction speed is described. (10) Correct 
so that the reproduction speed for the first portion of the shot which fills a formula 
may become below the rate upper limit Vmax (k) determined in the procedure 1303. 
Namely, like the shot SH2 of drawing 1 5 , after setting the reproduction speed VP for 
the first portion (k) as the value below the rate upper limit Vmax (k), it is determined in 



the second half that the reproduction speed VR of a part (k) will set display time of 
the whole rapid-traverse part to 2xTlen. However, the reproduction speed VR of a 
part (k) may exceed Vmax (k) in the second half. Moreover, in order to hold a rhythm, 
a stationary part is put in between parts in a part for the first portion, and the second 
half of a rapid-traverse part. 

[0100] The reproduction speed VP in the first half of a rapid-traverse part (k) is 

[0101]. 

[Equation 2] 

Vmax (K) , v CK) > Vmax ( K ) 
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It can determine "To be alike." Moreover, the reproduction speed VR in the second 
half of a rapid-traverse part (k) is [0102]. 
[Equation 3] 

VBCK) = L8H fc - VP(K)XIIe " (12) 

I i fin 



It can determine "To be alike." The procedure 409 stated above is ended after count 
of (11) and (12) types. 

[0103] In addition, when some shots which cannot grasp the contents may occur, a 
procedure 409 may be ended after termination of a procedure 1302. Moreover, when 
change of a shot is not predicted, reproduction speed V (k) may be determined, 
without using (9) types. 

[0104] It progresses to a procedure 410 after termination of the procedure 409 of 
drawing 4 . After from the procedure 410 to the procedure 412 unifies a similar shot, 
regards it as a shot group using correlation between the shots of time series and 
chooses from one shot group the partial dynamic image which consists of a frame 
image (for example, 1 second) of two or more time series, it is a procedure for realizing 
the image epitome approach which reproduces the partial dynamic image of time 
series one after another. First, correlation between the shots of time series is used, a 
similar shot is unified, and a shot group is created. Next, every one partial dynamic 
image is chosen from each shot group, and it is processed so that the description of a 
shot group may be specified. For example, in unifying a shot including a prototype 
object and creating a shot group, this prototype object serves as the description of a 
shot group. Moreover, when time amount length unifies an almost equal shot, this time 
amount length serves as the description of a shot group. 

[0105] When reproducing an epitome image, the processed partial dynamic image is 
reproduced one after another. In addition, the selected partial dynamic image may be 



reproduced directly, without performing processing. 

[0106] The epitome image created with the procedure 412 from the procedure 410 will 
be called a partial selection epitome image. 

[0107] A procedure 410 is processing which creates a shot group. The concrete 
actuation in a procedure 410 is described using drawing 16 . 

[0108] Since a shot is the unit of the contents, the frame image in one shot has one 
common description. For example, a person appears in all frame images at the shot 
which follows a person. Therefore, the contents of the shot can be represented using 
some frame images in a shot. In order to simplify processing, the number of sheets 
Nrep of a frame image which represents the contents is set up beforehand, and the 
contents of the shot are represented with this example using a Nrep sheet [ in a shot ] 
frame image (it is hereafter called the image between representation space-time). 
[0109] In a procedure 1601, the image representing Shot SHk is determined, the 
images Ik and j between representation space-time and (k regard it as one or more 
integers, and j regards it as integer) below or more 1 Nrep. However, as the decision 
approach of the image between representation space-time, they are various idea ****, 
such as the approach of choosing the Nrep sheet of a head part, and the approach of 
choosing the image of a fixed time interval. Moreover, as long as there is no limit in 
throughput, you may consider that all the frame images in a shot are the images 
between representation space-time. 

[01 10] The shot of the similar contents of time series is unified after termination of a 
procedure 1601 using the image between representation space-time, and it is 
regarded as a shot group. As an example of the shot of similar contents, there are a 
shot of a similar background, a shot of a similar photographic subject, etc. In this 
example, a shot is unified paying attention to the color of the body on a screen, and a 
motion. 

[0111] According to analysis of writers, by the image between representation 
space-time of the shot of similar contents, the body in which the color common to 
each image between representation space-time exists, and has a common color 
carries out a common motion. Although it is difficult to investigate whether two bodies 
which appear on a screen are carrying out the common motion, the change by motion 
of a body is easily detectable using a pixel change field, a motion vector, etc. which are 
mentioned later. Then, the image between representation space-time is processed 
and it divides into the field (it is hereafter called a motion area) influenced of change 
by motion of the body on a screen, and the remaining fields (it is hereafter called a 
quiescence field). At this time, the shot of similar contents satisfies the following 
common color ratio conditions. 

[0112] Common color ratio conditions: When calling the color which constitutes the 
quiescence field of the image between representation space-time, and the color which 
constitutes a motion area a quiescence configuration color and ********, 



respectively, the quiescence configuration color and ******** which it is under 
[ between representation space-time / image ] community exist, and a pixel with 
these colors accounts for the rate more than threshold thetashot by each image 
between representation space-time. 

[01 13] In addition, the example of the shot of similar contents which satisfies common 
color ratio conditions is shown below. 

Example 1 : a shot of a similar background like [ in case the shot photoed in the same 
location continues ]. two shot SHk-1 of the scenery of the poolside shown in drawing 
1 7 , and image Ik[ of SHk ] between representation space-time- in 1, 1, Ik, and 1 The 
quiescence configuration color D in rate =15%Ik~1 of the quiescence configuration 
color C in rate =20%Ik-1 of the quiescence configuration color B in rate =60%Ik-1 of 
the quiescence configuration color A in Ik— 1 and 1 and 1 and 1 and 1 comparatively 
=5%Ik, The quiescence configuration color C in Ik and 1 comparatively 80% The rate of 
the quiescence configuration color A in one = =15%Ik, Ik of the quiescence 
configuration colors A and C in which the rate in Ik— 1 [ of the quiescence 
configuration color E in one ] of the quiescence configuration colors A and C which 
become =5% comparatively and are common, and 1 is common sum =75%, the sum of 
the rate in one = since it becomes 95%, a common quiescence configuration color 
occupies 75% or more of the whole screen. 

[01 14] Example 2: a shot of a similar photographic subject like [ in case the shot of the 
same photographic subject continues ]. two shot SHk-1 of tailing of the automobile 
shown in drawing 18 , and image Ik[ of SHk ] between representation space-time- in 1, 
1, Ik, and 1 The quiescence configuration color Y in rate =10%Ik-1 of the quiescence 
configuration color X in rate =40%Ik-1 of the quiescence configuration color W in rate 
=40%Ik-1 of the quiescence configuration color V in Ik— 1 and 1 and 1 and 1 and 1 
comparatively =10%Ik, The quiescence configuration color W in Ik and 1 comparatively 
40% The rate of the quiescence configuration color Z in one = =35%Ik, Rate =10%Ik of 
the quiescence configuration color X in one, the rate of the quiescence configuration 
color Y in one = It becomes 1 5%. Ik of ******** W, X, and Y in which the rate in Ik— 1 of 
common ******** W, X, and Y and 1 is common sum =60%, the sum of the rate in one 
= since it becomes 60%, common ******** occupies 60% or more of the whole screen. 
[01 1 5] In the procedure 1 602 of drawing 16 , a motion area is detected from the image 
between representation space-time, and the image between representation 
space-time is divided into a motion area and a quiescence field. When a motion vector 
is used, a motion area comes to be shown in drawing 19 . Moreover, since the 
brightness of the pixel in the same location is almost equal when a body does not 
move between two frame images, when it considers that the next pixel change field is 
a motion area, a motion area comes to be shown in drawing 20 . 

[0116] pixel change field: — the frame images Ik and j in the image between 
representation space-time, and deflection after the 1 frame time between - MU 



images — " — the absolute value of the brightness difference of the pixel in the same 
location — threshold [ ] — set of pixel" which becomes thetaWI or more. 
[0117] A procedure 1603 is processing which asks for a quiescence configuration 
color and ********. The red component of a pixel, a green component, and a blue 
component will be hereafter called R, G, and B, respectively, and a procedure is 
explained. First, it asks for the histogram HS (c, SHk) of the 512 colors c of RGB eight 
gradation each in the quiescence field of the image between representation 
space-time of Shot SHk. Similarly, it asks for the histogram HM (c, SHk) of a motion 
area. Next, in order that the number of pixels may consider that few colors are noises 
and may except them, number of minimum pixels thetaH of a configuration color is set 
up, and it asks for the quiescence configuration color HSVk and ******** HMVk (k is 
one or more integers) using a degree type. 
[0118] 

HSVk= {c|HS(c, SHk) XhetaH} (13) 

HMVk= {c|HM(c, SHk) XhetaH} (14) 

Below, the shot of the head of a shot group is described to be SHtop (for top to be 
one or more integers). A procedure 1604 to the procedure 1606 is the processing 
which investigates the shot of the time series after this shot SHtop, and investigates 
the shot to where fulfills common color ratio conditions. 

[01 1 9] In a procedure 1604, it supposes that the quiescence configuration color which 
is common between shot SHtop-SHtop+m, and ******** are called the quiescence 
common colors HSCtop and m and the ** common colors HMCtop and m (top and m 
are one or more integers), respectively, and they are calculated using a degree type. 
[0120] 
[Equation 4] 

HS C top,m= {c | cEHSVlopfl HS V top+l fl - D HSVtop+m } 

C 1 5} 

UUC tap>m= {c I C6HMV tppPHMVlop+i n H HMVtop+m } 

(16) 

In a procedure 1605, it supposes that "the rate that a pixel with the quiescence 
common colors HSCtop and m and the ** common colors HMCtop and m occupies in 
the image between representation space-time of Shot SHk" is called the image 
common color ratio AMC (k, top, m), and calculates using a degree type. 
[0121] 
[Equation 5] 
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(17) 



Formula which expresses formation of common color ratio conditions with a 

procedure 1606 AMC(k, top, m) >=thetashot, top<=k<=top+m (18) 

When materialized to ** and all k, it is considered that shot SHtop-SHtop+m satisfies 
common color ratio conditions. In order to investigate the shot to where fulfills 
common color ratio conditions, when (18) types are materialized to all k, after adding 1 
to m, it returns to a procedure 1604. When that is not right, it progresses to a 
procedure 1607. 

[0122] In this example, it asks for the tail of a shot group using the image common 
color ratio between two shots of time series. First, the image common color ratios 
AMC (k, k1 ) and AMC (k+ 1k, 1 ) (k is an integer below top+m more than top) which (1 7) 
types defined are calculated in a procedure 1607. Former AMC (k, k1) expresses "the 
rate that two shots SHk, and the quiescence common color and ** common color of 
SHk+1 occupy in the image between representation space-time of the front shot 
SHk." Latter AMC (k+ 1k, 1) expresses "the rate that two shots SHk, and the 
quiescence common color and ** common color of SHk+1 occupy in the image 
between representation space-time of next shot SHk+1." 

[0123] By the procedure 1608, it is the minimum value of an image common color ratio. 
SS(k) =min (AMC (k, k1), AMC (k+ 1k, 1)) (19) 

It calculates and is regarded as the priority for integration of Shot SHk and SHk+1. 
[0124] In a procedure 1609, it asks for the tail of a shot group out of shot 
SHtop-SHtop+m -1 which fulfills common color ratio conditions, and its following shot 
SHtop+m. Formula which expresses that a priority is min out of the priorities SS 
(top)-SS (top+m -1) of integration between these shots SS(kmin) <=SS (k), 
top<=k<=top+m -1 (20) 

******** SS (kmin) is calculated and it is considered like drawing 21 (a) and the 
example of (b) that the shot SHkmin from which a priority becomes min is the tail of a 
shot group, ((b) drawing 21 (a) and 2nd shot) And the procedure 1610 stated above is 
ended. 

[0125] In addition, the approaches of asking for a shot group are various idea 

For example, at the shot of time series, since the contents are similar when time 

amount length is almost equal, it can be regarded as one shot group. Therefore, you 

may ask for a shot group using a different approach from this example. 

[0126] Moreover, it may consider after a procedure 1606 that tail SHk+m -1 of the 

shot which fulfills common color ratio conditions is the tail of a shot group, without 



progressing to a procedure 1607 f and a procedure 410 may be ended. 
[0127] It progresses to a procedure 411 after termination of the procedure 410 of 
drawing 4 . Every one partial dynamic image is chosen from each shot group in a 
procedure 411. However, the selection approach of a partial dynamic image has 
various kinds of approaches, such as a head part of the head shot of a shot group, and 
a head part of the shot representing the contents of the shot group. Moreover, various 
kinds of approaches, such as the approach of making it into a fixed length and a 
method of changing time amount length according to the audio contents, can be 
considered also about the setting approach of the time amount length of a partial 
dynamic image. 

[0128] In addition, in a procedure 41 1, although the partial dynamic image was chosen, 
you may choose one static image from each shot group at a time. In this case, it will 
become an epitome image if the selected static image is displayed one after another. 
Moreover, the selected static image may be reduced and you may indicate by list. A 
list display result can be treated on a par with an epitome image. 
[0129] It progresses to a procedure 412 after termination of a procedure 411. A 
procedure 412 is processing for specifying the description of a shot group, and carries 
out processing processing of the partial dynamic image chosen in the procedure 41 1. 
[0130] The shot contained in a shot group fills common color ratio conditions with this 
example. That is, the quiescence common color and ** common color for which each 
shot which constitutes a shot group was asked in the procedure 1607 exist. 
[0131] Since a quiescence common color and a ** common color express the color of 
the body which exists in common with a shot group, it can be considered that they are 
the description of a shot group. For example, if it is the shot of a similar background, 
the color of a background is expressed, and if it is the shot of a similar photographic 
subject, the color of a photographic subject is expressed. So, in a procedure 412, the 
pixel of a quiescence common color and a ** common color is similarly displayed as 
usual, and pixels other than this are expressed as half brightness. In addition, as long 
as it emphasizes a quiescence common color and a ** common color, what kind of 
processing art may be used, such as making the brightness of the pixel except a 
quiescence common color and a ** common color into zero. 

[0132] Moreover, the processing art of an image is decided in accordance with the 
integrated approach of a shot, and various variations exist. For example, when time 
amount length unifies an almost equal shot, the processing art which displays the 
mean-time length of a shot on the partial dynamic-image bottom can be considered 
like drawing 22 . 

[0133] It returns to a procedure 402 after termination of a procedure 412. In addition, 
when the description of a shot group does not need to be specified, you may return to 
a procedure 402 after termination of a procedure 41 1, without performing a procedure 
412. 



[0134] The above is one example of image epitome processing of the 1st computer 5 
in drawing 1 , and is detailed explanation of the procedure 201 of drawing 2 . 
[0135] The procedure 202 of drawing 2 is performed after termination of image 
epitome processing. The 1st calculating machine 5 in drawing 1 controls videodisk 
equipment 1 and VTR2 by the procedure 202, and reproduces in it a 
processing-object image and the image summarized in the procedure 201. The 
reproduced image is compressed by the MPEG method in the image compression 
equipment 4 of drawing 1 , and is memorized by the file server 6. 
[0136] However, since the procedure 201 of this example creates the playback 
approach of an epitome image as shown in drawing 3 , without creating the 
summarized image, it is controlling videodisk equipment 1 and VTR2 according to the 
playback approach of this epitome image, and performs playback of an epitome image. 
[0137] In addition, it is not necessary to necessarily compress by the MPEG method, 
and you may compress by other compression methods, such as a JPEG method. 
Moreover, not only the approach of compressing an image directly about the store 
method of an epitome image, as already stated but other approaches are various idea 

[0138] Below, epitome image regeneration of the 2nd computer 7 of drawing 1 which is 
concrete actuation of the procedure 203 in drawing 2 is described. 
[0139] Drawing 23 is the flow chart of one example of epitome image regeneration of 
the 2nd calculating machine 7 in drawing 1 . 

[0140] The graphic display approach is chosen in a procedure 2301. However, much 
alternative can be considered. For example, what is necessary is just to choose 
"standard speed playback of a processing-object image" to see the contents of the 
image in detail. The image which a user wants to see at the time of graphic display is 
reproduced at an ordinary rate. 

[0141] Moreover, what is necessary is just to choose the "section gear change 
epitome image" created by processing from the procedure 405 of drawing 4 to a 
procedure 408 to grasp the contents focusing on actuation of a photographic subject. 
The whole image is reproduced seeing subjectively and making it reproduction speed 
enter in tolerance at the time of graphic display. 

[0142] In addition, there is individual difference in subjectivity. Therefore, a user may 
choose the reproduction speed decision approach according to his subjectivity. 
Instead of performing the following three processings, only the part may be performed 
or, specifically, the processing "which is the part of the fine image of a texture and 
makes reproduction speed slow relatively" may be added. 

[0143] - the procedure 408 for making reproduction speed late relatively at procedure 
406 and the short-time merit shot for making reproduction speed late relatively in 
procedure 405 and the high-speed operation section for making reproduction speed 
quick relatively in the similar section for a long time — what is necessary is just to 



choose the "rhythm presentation epitome image created with the procedure 409 of 
drawing 4 to know further the time flow of the contents included in an image The shot 
which summarized the part of the same contents at the time of graphic display is 
reproduced one after another. 

[0144] What is necessary is just to choose the "partial selection epitome image" 
created by processing from the procedure 410 of drawing 4 to a procedure 412 to see 
the part of different contents as much as possible little by little. The shot group which 
summarized the shot of the contents from which time series is similar at the time of 
graphic display is reproduced one after another. 

[0145] In the procedure 2302 of drawing 23 , according to the selection result by the 
procedure 2301, a required image is called from the file server 6 of drawing 1 , and the 
image is reproduced. However, when only the frame number information for epitome 
image creation is saved at the file server 6, it reproduces according to frame number 
information, summarizing a processing-object image. Moreover, a user may interrupt 
playback at the favorite time, specifying "to [ from which frame ] which frame it 
reproduces", specifying "from which frame it reproduces", and looking at an image 
before image reproduction. 

[0146] In addition, what is necessary is just to decide the flow of the image epitome 
processing shown in drawing 4 in accordance with the alternative of a procedure 2301. 
For example, when a "rhythm presentation epitome image" is not included in the 
alternative of a procedure 2301, it is not necessary to perform the procedure 409 of 
drawing 4 . Similarly, when a "section gear change epitome image" is not included, it is 
not necessary to perform from the procedure 405 of drawing 4 to the procedure 408, 
and when a "partial selection epitome image" is not included, it is not necessary to 
perform from the procedure 410 of drawing 4 to the procedure 412. 
[0147] Moreover, although this example showed the example which reproduces the 
summarized image and a processing-object image on the 2nd calculating machine 7 of 
drawing 1 , you may reproduce on the alien machine connected to the file server 6. the 
case of drawing 1 where may reproduce on the 1st calculating machine 5 and two or 
more calculating machines are connected to the file server 6 — those the count of all 
— it is on board and you may reproduce. 

[0148] As mentioned above, according to the section gear change epitome image of 
this example, the time amount section when an image similar over beyond fixed time 
amount continues is detected, and it is regarded as the long duration similar section, 
and by reproducing the long duration similar section quickly relatively, since an image 
changes one after another for a short time, section <TXF FR=0001 HE=250 WI=080 
LX=0200 LY=0300> part" with slow reproduction speed disappears on "subjectivity 
target. Moreover, the time amount section containing the body which moves at high 
speed is detected, and it is regarded as the high-speed operation section, and since 
the body which runs by making late relatively reproduction speed of the high-speed 



operation section at high speed stops existing, it is a part with quick reproduction 
speed subjectively" decreases. Furthermore, since each contents are displayed 
beyond fixed time amount by preparing a minimum in the display time length of each 
shot after summarizing an image for every contents and dividing into two or more 
shots, "it is a part with quick reproduction speed subjectively" disappears. Thus, 
compared with a conventional method, fatigue of a user is mitigated by reproducing 
the whole image, seeing subjectively and making it reproduction speed enter in 
tolerance. Moreover, contents centering on actuation of a photographic subject can 
be grasped. 

[0149] Moreover, with the rhythm presentation epitome image of this example, after 
setting an upper limit as the display time length of each shot, the shot of time series is 
reproduced one after another, determining that reproduction speed will satisfy at least 
one side of the following two conditions, and reminding a rhythm. 
[0150] Rhythm conditions: The boundary of a shot and the rhythm of the period set up 
beforehand have correlation. 

[0151] Contents conditions : When the rate upper limit is set up so that it may 
guarantee "the part reproduced at the rate of below the rate upper limit which is the 
criteria of reproduction speed decision can surely grasp the contents", a part of [ at 
least ] reproduction speed of each shot becomes below a rate upper limit. 
[0152] In summarizing an image as satisfying rhythm conditions, since change of a 
shot can be predicted, it can see, without overlooking no shots. 

[0153] In summarizing an image as satisfying contents conditions, a user can grasp 

some of the contents at least in all shots. Since it is the shot which summarized the 

part of the same contents, the contents of all the shots can be grasped. 

[0154] However, since change of a shot cannot be predicted when not satisfying 

rhythm conditions, an oversight of a shot occurs. When not satisfying contents 

conditions, the contents of some shots cannot be grasped. It is desirable that it is 

satisfied with coincidence of rhythm conditions and contents conditions. 

[0155] Furthermore, with the partial selection epitome image of this example, after 

unifying a similar shot and creating a shot group using correlation between the shots 

of time series, the partial dynamic image chosen from each shot group is reproduced 

one after another. By using this epitome image, the part of different contents as much 

as possible can be seen efficiently. 

[0156] 

[Effect of the Invention] As mentioned above, this invention detects the time amount 
section when an image similar to the 1st over beyond fixed time amount continues, 
considers for a long time that it is the similar section, by reproducing the similar 
section quickly relatively for a long time, can change an image one after another in a 
short time, and can extinguish "it is a part with slow reproduction speed subjectively." 
Moreover, the time amount section of the image containing the body which moves at 



high speed is detected, and it is regarded as the high-speed operation section, and by 
making late relatively reproduction speed of the high-speed operation section, the 
body which moves at high speed can be extinguished and it can decrease "it is a part 
with quick reproduction speed subjectively." Furthermore, after summarizing the part 
of the same contents to a shot by preparing a minimum in the display time length of 
each shot, each contents can be displayed beyond fixed time amount, and it can 
extinguish "it is a part with quick reproduction speed subjectively." Thus, compared 
with a conventional method, fatigue of a user is mitigated by reproducing the whole 
image, seeing subjectively and making it reproduction speed enter in tolerance. 
Moreover, contents centering on actuation of a photographic subject can be grasped. 
[0157] Moreover, the shot of time series is reproduced one after another, determining 
that reproduction speed will satisfy at least one side of the following two conditions, 
and making the 2nd recollect a rhythm, after setting an upper limit as the display time 
length of each shot. 

[0158] Rhythm conditions: The boundary of a shot and the rhythm of the period set up 
beforehand have correlation. 

[0159] Contents conditions : When the rate upper limit is set up so that it may 
guarantee "the part reproduced at the rate of below the rate upper limit which is the 
criteria of reproduction speed decision can surely grasp the contents", a part of [ at 
least ] reproduction speed of each shot becomes below a rate upper limit. 
[0160] In summarizing an image as satisfying rhythm conditions, since change of a 
shot can be predicted, it can see, without overlooking no shots. 

[0161] In summarizing an image as satisfying contents conditions, a user can grasp 

some of the contents at least in all shots. Since it is the shot which summarized the 

part of the same contents, the contents of all the shots can be grasped. 

[0162] However, since change of a shot cannot be predicted when not satisfying 

rhythm conditions, an oversight of a shot occurs. When not satisfying contents 

conditions, the contents of some shots cannot be grasped. It is desirable that it is 

satisfied with coincidence of rhythm conditions and contents conditions. 

[0163] After unifying a similar shot and creating [ 3rd ] a shot group using correlation 

between the shots of time series furthermore, the partial dynamic image chosen from 

each shot group is reproduced one after another. By using this epitome image, the 

part of different contents as much as possible can be seen efficiently. 
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[Brief Description of the Drawings] 

[Drawing 1] The whole image epitome equipment system chart in one example of this 
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invention 

[Drawing 2] The flow chart of actuation of the image epitome equipment in this 
example 

[Drawing 3] The conceptual diagram of an expression of the epitome image by the file 
format in this example 

[Drawing 4] The flow chart of the image epitome processing in this example 
[Drawing 5] The flow chart of the long duration similar section detection processing in 
this example 

[Drawing 6] Drawing showing the example of creation of the subregion in this example 
[Drawing 7] Drawing showing the part which carries out loose image change of the 
image in this example 

[Drawing 8] The flow chart of the high-speed operation section detection processing 
in this example 

[Drawing 9] Drawing showing the case where the body on the screen in this example 
moves to a high speed 

[Drawing 10] Drawing showing the comparison of a pixel only using (6) types in this 
example 

[Drawing 1 1] Drawing showing the comparison of the pixel of a location p in (6) in this 
example, and (7) types 

[Drawing 12] Drawing showing the field which performs high-speed operation pixel 
detection in this example 

[Drawing 13] The flow chart of the rhythm presentation epitome processing in this 
example 

[Drawing 14] The conceptual diagram of the playback approach which fulfills the 
rhythm conditions in this example 

[Drawing 15] The conceptual diagram of the playback approach of the rhythm 
presentation epitome image in this example 

[Drawing 16] The flow chart of the shot integrated processing in this example 
[Drawing 17] The conceptual diagram of the shot of the similar background in this 
example 

[Drawing 18] The conceptual diagram of the shot of the similar photographic subject 
in this example 

[Drawing 19] The conceptual diagram of the motion area in this example 

[Drawing 20] The conceptual diagram of the motion area in this example 

[Drawing 21] The conceptual diagram of the tail decision processing in this example 

[Drawing 22] The conceptual diagram of the processing approach of the partial 

dynamic image in this example 

[Drawing 23] The flow chart of the epitome image reproduction processing in this 
example 
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1 Videodisk Equipment 

2 VTR 

3 Frame Memory 
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